{Core} Fix 404 in long running delete operation by fengzhou-msft · Pull Request #15884 · Azure/azure-cli

fengzhou-msft · 2020-11-12T07:35:52Z

Description

In the response of DELETE long running operation, some service (observed in some corner case of servicebus, update: confirmed in swagger review session that more services have this behavior) may return 404 while GETting operation results when the resource is successfully deleted (normally it should return 200 with no response body).

To handle the 404 case, this PR adds similar logic to the error handling in wait --deleted command:

azure-cli/src/azure-cli-core/azure/cli/core/commands/arm.py

Lines 700 to 702 in 05e3b63

    
           if getattr(ex, 'status_code', None) == 404: 
        
               if wait_for_deleted: 
        
                   return None

Testing Guide

Without this PR, azdev test test_sb_alias --live will fail when deleting the secondary namespace.
With this PR, it can succeed.

History Notes

[Component Name 1] BREAKING CHANGE: az command a: Make some customer-facing breaking change.
[Component Name 2] az command b: Add some customer-facing feature.

This checklist is used to make sure that common guidelines for a pull request are followed.

The PR title and description has followed the guideline in Submitting Pull Requests.
I adhere to the Command Guidelines.
I adhere to the Error Handling Guidelines.

yonzhan · 2020-11-12T10:03:22Z

Core

src/azure-cli-core/azure/cli/core/commands/__init__.py

jiasli · 2021-02-08T06:20:37Z

Is GET called on the service itself or the operation status (Azure-AsyncOperation/Location)?

jiasli · 2021-02-08T06:24:09Z

src/azure-cli-core/azure/cli/core/commands/__init__.py

        except ClientException as client_exception:
            from azure.cli.core.commands.arm import handle_long_running_operation_exception
            self.cli_ctx.get_progress_controller().stop()
+            if getattr(client_exception, 'status_code', None) == 404 and 'delete' in self.cli_ctx.data['command']:


Even if we add this hack, Python SDK and its users will still suffers from this non-standard behavior.

Do we have an open issue in https://github.com/Azure/azure-rest-api-specs to track this?

I opened an issue in our repo #15906 for service team to deal with.

Based on a disucssion with swagger reviewers, there are some existing services using 404 in LRO. For new swaggers, there should be validation tasks to catch it but for existing ones as long as the json is not modified with PRs, it will not be caught.

I still think we should force service team to fix it, instead of patching it from our side, otherwise Python SDK and its users will still suffers from this non-standard behavior.

fengzhou-msft · 2021-02-08T09:17:29Z

Is GET called on the service itself or the operation status (Azure-AsyncOperation/Location)?

The error is thrown during long running operation, so I think it's called on the operation status.

        # Delete Namespace - primary
>       self.cmd('servicebus namespace delete --resource-group {rg} --name {namespacenameprimary}')

azure-cli/src/azure-cli/azure/cli/command_modules/servicebus/tests/latest/test_servicebus_alias_commands.py:163: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
azure-cli/src/azure-cli-testsdk/azure/cli/testsdk/base.py:167: in cmd
    return execute(self.cli_ctx, command, expect_failure=expect_failure).assert_with_checks(checks)
azure-cli/src/azure-cli-testsdk/azure/cli/testsdk/base.py:238: in __init__
    self._in_process_execute(cli_ctx, command, expect_failure=expect_failure)
azure-cli/src/azure-cli-testsdk/azure/cli/testsdk/base.py:301: in _in_process_execute
    raise ex.exception
env/lib/python3.8/site-packages/knack/cli.py:233: in invoke
    cmd_result = self.invocation.execute(args)
azure-cli/src/azure-cli-core/azure/cli/core/commands/__init__.py:659: in execute
    raise ex
azure-cli/src/azure-cli-core/azure/cli/core/commands/__init__.py:722: in _run_jobs_serially
    results.append(self._run_job(expanded_arg, cmd_copy))
azure-cli/src/azure-cli-core/azure/cli/core/commands/__init__.py:715: in _run_job
    six.reraise(*sys.exc_info())
env/lib/python3.8/site-packages/six.py:703: in reraise
    raise value
azure-cli/src/azure-cli-core/azure/cli/core/commands/__init__.py:704: in _run_job
    result = LongRunningOperation(cmd_copy.cli_ctx, 'Starting {}'.format(cmd_copy.name))(result)
azure-cli/src/azure-cli-core/azure/cli/core/commands/__init__.py:988: in __call__
    handle_long_running_operation_exception(client_exception)

houk-ms · 2021-04-22T06:48:52Z

Soft-delete and purge operation on managed-hsm also suffer from this bad behavior. It just returns 404 when the delete operation completes.

I think we should have this logic supported in CLI core to handle these cases. Maybe also add some log info to expose these cases. Of course, at the same time, we should report these bad behiviors to service team.

houk-ms

LGTM

fix 404 in LRO

ef799b4

fengzhou-msft requested review from Juliehzl, arrownj, haroldrandom, jiasli, jsntcy and v-Ajnava as code owners November 12, 2020 07:35

arrownj approved these changes Nov 13, 2020

View reviewed changes

Juliehzl reviewed Nov 13, 2020

View reviewed changes

src/azure-cli-core/azure/cli/core/commands/__init__.py Show resolved Hide resolved

yonzhan assigned fengzhou-msft Nov 27, 2020

Merge branch 'dev' into fix_delete_lro

0070732

fengzhou-msft requested review from evelyn-ys and houk-ms as code owners February 8, 2021 04:40

jiasli reviewed Feb 8, 2021

View reviewed changes

Merge branch 'dev' into fix_delete_lro

059532d

houk-ms approved these changes Apr 22, 2021

View reviewed changes

yungezz merged commit 2ff95ab into Azure:dev Apr 22, 2021

houk-ms mentioned this pull request Apr 23, 2021

{Core} Work around long-running delete or purge operation with 404 status code #17833

Merged

3 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

{Core} Fix 404 in long running delete operation#15884

{Core} Fix 404 in long running delete operation#15884
yungezz merged 3 commits intoAzure:devfrom
fengzhou-msft:fix_delete_lro

fengzhou-msft commented Nov 12, 2020 •

edited

Loading

Uh oh!

yonzhan commented Nov 12, 2020

Uh oh!

Uh oh!

jiasli commented Feb 8, 2021

Uh oh!

jiasli Feb 8, 2021

Uh oh!

jiasli Feb 8, 2021

Uh oh!

fengzhou-msft Feb 8, 2021

Uh oh!

fengzhou-msft Feb 8, 2021

Uh oh!

jiasli Feb 8, 2021

Uh oh!

fengzhou-msft commented Feb 8, 2021

Uh oh!

houk-ms commented Apr 22, 2021 •

edited

Loading

Uh oh!

houk-ms left a comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

7 participants

	if getattr(ex, 'status_code', None) == 404:
	if wait_for_deleted:
	return None

Conversation

fengzhou-msft commented Nov 12, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

yonzhan commented Nov 12, 2020

Uh oh!

Uh oh!

jiasli commented Feb 8, 2021

Uh oh!

jiasli Feb 8, 2021

Choose a reason for hiding this comment

Uh oh!

jiasli Feb 8, 2021

Choose a reason for hiding this comment

Uh oh!

fengzhou-msft Feb 8, 2021

Choose a reason for hiding this comment

Uh oh!

fengzhou-msft Feb 8, 2021

Choose a reason for hiding this comment

Uh oh!

jiasli Feb 8, 2021

Choose a reason for hiding this comment

Uh oh!

fengzhou-msft commented Feb 8, 2021

Uh oh!

houk-ms commented Apr 22, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

houk-ms left a comment

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

7 participants

fengzhou-msft commented Nov 12, 2020 •

edited

Loading

houk-ms commented Apr 22, 2021 •

edited

Loading